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DETAILED ACTION 

Response to Amendment 

1 . This is in response to an amendment/response filed on August 20 th , 2009. 

2. Claims 1 and 22 have been amended. 

3. No claims have been cancelled. Claims 3 and 8 were previously cancelled. 

4. No claims have been added. 

5. Claims 1, 2, 4-7, and 9-29 are currently pending. 

Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

8. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
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claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S. C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

9. Claims 1, 2, 4, 6, 7, and 9-29 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Saito (U.S. Patent 6,987,767) in view of Maclnnis et al. (U.S. Patent 6,853,385). 

For claim 1, Saito discloses a method of forming an output media stream to be 
transmitted during a communication session from a portable communication device wherein said 
output media stream comprises signals of a first media type, the method comprising: 

generating in real time a first media stream in the portable communication device (see 
column 9 lines 14-24 and figure 4, which recite microphone 3 land camera 33 that generate a 
real-time media stream as input to multiplexer 22), 

combining in real time the first media stream with a second media stream to form the 
output media stream (see column 9 lines 62-67, column 10 lines 1-2, and figure 1, which recite a 
multiplexer 22 that combines a media stream generated by microphone 31 and camera 33 with 
media streams of various encoding units), 

wherein combining comprises combining signals of the first media type from the first 
media stream with signals of the first media type from the second media stream (see column 4 
lines 20-27 and 50-56, which recite combining the video streams of the encoding units and 
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further combining the audio streams of the encoding units using the TS multiplexing section 
310), 

transmitting said output media stream (see column 9 lines 25-31, which recite 
transmitting the multiplexed stream using transmission circuit TX 15). 

Saito discloses all the subject matter of the claimed invention with the exception wherein 
combining media streams specifically comprises superposing signals of the first media type from 
the first media stream on signals of the first media type from the second media stream to produce 
the output media stream, wherein the output media stream comprises portions of the first and 
second media streams which are configured to be presented in a substantially simultaneous time. 
However, Maclnnis et al. from the same or similar fields of endeavor disclose a system for 
combining audio and video transport streams into blended graphics or two simultaneous in-band 
streams in a picture-in-picture format that are presented in a substantially simultaneous time (see 
column 1 lines 60-67 and column 2 lines 1-16). Thus, it would have been obvious to the person 
of ordinary skill in the art at the time of the invention to use the system that superposes media 
streams as taught by Maclnnis et al. with the communication device configured to form an output 
media stream as taught by Saito. The MPEG transport streams as taught by Saito can be 
superimposed into an output stream using the system as taught by Maclnnis et al. The 
motivation for using the system that superposes media streams as suggested by Maclnnis et al. 
with the communication device configured to form an output media stream as taught by Saito is 
to improve the performance of the system by transmitting information from multiple media 
streams as a single superposed media stream. 
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For claim 2, Saito discloses disclose a method for forming an output media stream 
wherein said output media stream comprises signals of a second media type (see column 9 lines 
25-31, which recite a multiplexed transport stream containing both audio and video data). 

For claim 4, Saito discloses a method of forming an output media stream that further 
comprises establishing a connection with another device (see column 8 lines 23-50 and column 9 
lines 14-31, which recite a transmission circuit 15 used to establish a connection with another 
device). 

For claim 6, Saito discloses a method for forming an output media stream in which at 
least one of the steps is dependent on input data from a user of said portable communication 
device (see figure 4, which recite an input section 35 that provides input for the main control 
portion 21 that is responsible for forming the multiplexed media streams). 

For claim 7, Saito discloses disclose a method for forming an output media stream 
wherein combining comprises combining signals of the first media type from the first media 
stream with signals of a second media type from the second media stream (see column 4 lines 
20-27 and 50-56, which recite combining the video streams of multiple encoding units and 
further combining the audio streams of the multiple encoding units using TS multiplexing section 
310). 

For claim 9, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises combining signals of a second media type from the first 
media stream with the signals from the second media stream (see column 4 lines 20-27 and 50- 
56, which recite combining the video streams of multiple encoding units and further combining 
the audio streams of the multiple encoding units using TS multiplexing section 310). 
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For claim 10, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises combining signals from the first media stream with signals 
of the second media type from the second media stream (see column 4 lines 20-27 and 50-56, 
which recite combining the video streams of multiple encoding units and further combining the 
audio streams of the multiple encoding units using TS multiplexing section 310). 

For claim 11, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises combining signals of the second media type from the first 
media stream with signals from the second media stream (see column 4 lines 20-27 and 50-56, 
which recite combining the video streams of multiple encoding units and further combining the 
audio streams of the multiple encoding units using TS multiplexing section 310). 

For claim 12, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises delaying, prior to combining, signals of one media type of 
the second media stream in relation to the other media type of signals of the same stream to 
provide synchronized signals from the second media stream within the output media stream (see 
column 7 lines 23-31, which recite delaying signals of one media type by shifting the value of the 
Presentation Time Stamp PTS value to a later time to provide synchronized signals). 

For claim 13, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises independently combining signals of the first media type 
and signals of the second media type (see column 4 lines 20-27 and 50-56, which recite 
combining the video streams of multiple encoding units and further combining the audio streams 
of the multiple encoding units using TS multiplexing section 310). 
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For claim 14, Saito discloses disclose a method for forming an output media stream 
wherein combining further comprises delaying signals of one media type within the output media 
stream, in relation to the other media type of signals of the same stream to provide synchronized 
signals from the first media stream within the output media stream (see column 7 lines 23-31, 
which recite delaying signals of one media type that is part of the multiplexed transport stream 
by shifting the value of the Presentation Time Stamp PTS value to a later time to provide 
synchronized signals). 

For claim 15, Saito discloses disclose a method for forming an output media stream 
wherein the signals of the first media type are audio signals so that the signals of the first media 
type from the first media stream comprise first audio signals and the signals of the first media 
type from the second media stream comprise second audio signals (see column 4 lines 20-27 and 
50-56, which recite combining the video streams of the encoding units and further combining the 
audio streams of the encoding un its using the TS multiplexing section 310), wherein combining 
further comprises superposing the first and second audio signals of the first and second media 
streams (see column 4 lines 57-67 and figure 6, which recite superposing the audio streams into 
a single transport stream by multiplexing individual partitioned PES portions of the parse-PES 
generation sections 612 to 6n2 into a single transport stream) . 

For claim 16, Saito discloses disclose a method for forming an output media stream 
wherein superposing comprises weighting properties of the audio signals from the first media 
stream and the second media stream (see column 4 lines 57-67 and figure 6, which recite 
superposed audio streams with weighting properties based upon the amount of partitioned PES 
portions that each audio stream provide for multiplexing to the TS multiplexing section 310, 
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since the more partitioned PES portions are provided for multiplexing, the more data from that 
stream becomes part of the multiplexed transport stream). 

For claim 17, Saito discloses disclose a method for forming an output media stream 
wherein the signals of the first media type are image signals so that the signals of the first media 
type from the first media stream comprise first image signals and the signals of the first media 
type from the second media stream comprise second image signals (see column 4 lines 20-27 and 
50-56, which recite combining the video streams of the encoding units and further combining the 
audio streams of the encoding units using the TS multiplexing section 310), wherein combining 
further comprises blending the first and second image signals of the first and second media 
streams (see column 4 lines 57-67 and figure 6, which recite superposing the video streams into 
a single transport stream by multiplexing individual partitioned PES portions of the parse-PES 
generation sections 611 to 6nl into a single transport stream). 

For claim 18, Saito discloses disclose a method for forming an output media stream 
wherein blending comprises weighting properties of the image signals from the first media 
stream and the second media stream (see column 4 lines 57-67 and figure 6, which recite 
superposed videostreams with weighting properties based upon the amount of partitioned PES 
portions that each audio stream provide for multiplexing to the TS multiplexing section 310, 
since the more partitioned PES portions are provided for multiplexing, the more data from that 
stream becomes part of the multiplexed transport stream). 

For claim 19, Saito discloses disclose a method for forming an output media stream 
wherein weighting properties includes varying the proportion of signals from the first media 
stream in relation to the proportion of signals from the second media stream (see column 4 lines 
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57-67 and figure 6, which recite superposed audio streams with varying proportion of signals 
since as more partitioned PES portions are provided for multiplexing by the audio stream, less of 
the resulting multiplexed transport stream comprises audio from the other audio streams). 

For claim 20, Saito discloses disclose a method for forming an output media stream 
wherein weighting properties is dependent on input data of a user of said portable 
communication device (see figure 4, which recite microphone 31, camera 32, and key input 
section 35 of input section 3 which takes input data of a user, wherein the weighting properties 
of the input data depends on the amount of input data provided by the user). 

For claim 21, Saito discloses disclose a method for forming an output media stream 
wherein varying said proportions comprises varying of each proportion within the range between 
0 and 100% (see column 4 lines 57-67 and figure 6, which recite superposed audio streams with 
weighting properties based upon the amount of partitioned PES portions that each audio stream 
provide for multiplexing to the TS multiplexing section 310, wherein an audio stream may 
provide all of the audio data to be multiplex or no data to multiplex). 

For claim 22, Saito discloses a portable communication device configured to form an 
output media stream to be transmitted during a communication session from said portable 
communication device, wherein said output media stream comprises signals of a first media type, 
said portable communication device comprising: 

at least one generating unit configured to generate a first media stream (see column 9 
lines 14-24 and figure 4, which recite microphone 3 land camera 33 that generate a real-time 
media stream as input to multiplexer 22), 
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a first combining unit, connected to said generating unit, wherein the first combining unit 
is configured to combine in real time the first media stream with a second media stream (see 
column 9 lines 62-67, column 10 lines 1-2, and figure 1, which recite a multiplexer 22 that 
combines a media stream generated by microphone 31 and camera 33 with media streams of 
various encoding units), 

wherein the first combining unit is configured to combine signals of the first media type 
from the first media stream with signals of the first media type from the second media stream to 
form the output media stream (see column 4 lines 20-27 and 50-56, which recite TS multiplexing 
section 310 of multiplexer 22 that combines the video streams of the encoding units and further 
combines the audio streams of the encoding units), 

a control unit configured to control the generating unit and the combining unit, in 
dependence of user input (see figure 4, which recite a main control portion 21 that receives user 
input from input section 35), and 

a transmitter configured to transmit said output media stream (see column 9 lines 25-31, 
which recite transmitting the multiplexed stream using transmission circuit TX 15). 

Saito discloses all the subject matter of the claimed invention with the exception wherein 
combining media streams specifically comprises superposing signals of the first media type from 
the first media stream on signals of the first media type from the second media stream to produce 
the output media stream, wherein the output media stream comprises portions of the first and 
second media streams which are configured to be presented in a substantially simultaneous time. 
However, Maclnnis et al. from the same or similar fields of endeavor disclose a system for 
combining audio and video transport streams into blended graphics or two simultaneous in-band 
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streams in a picture-in-picture format that are presented in a substantially simultaneous time (see 
column 1 lines 60-67 and column 2 lines 1-16). Thus, it would have been obvious to the person 
of ordinary skill in the art at the time of the invention to use the system that superposes media 
streams as taught by Maclnnis et al. with the communication device configured to form an output 
media stream as taught by Saito. The MPEG transport streams as taught by Saito can be 
superimposed into an output stream using the system as taught by Maclnnis et al. The 
motivation for using the system that superposes media streams as suggested by Maclnnis et al. 
with the communication device configured to form an output media stream as taught by Saito is 
to improve the performance of the system by transmitting information from multiple media 
streams as a single superposed media stream. 

For claim 23, Saito discloses a portable communication device configured to form an 
output media stream, wherein the first combining unit is configured to combine signals of the 
first media type of both the first and the second media streams, wherein the output media stream 
comprises signals of the first media type and a second media type, wherein the portable device 
further comprises: 

a second combining unit configured to combine signals of the second media type of the 
first media stream and signals of the second media type of the second media stream by using the 
second combining unit (see column 4 lines 20-27 and 50-56, which recite TS multiplexing 
section 310 of multiplexer 22 that combines the video streams of the encoding units and further 
combines the audio streams of the encoding units). 

For claim 24, Saito discloses a portable communication device configured to form an 
output media stream, further comprising a memory unit configured to provide storage for the 
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second media stream (see column 9 lines 1-4, which recite memory 26 that stores video and 
audio data). 

For claim 25, Saito discloses a portable communication device configured to form an 
output media stream, further comprising a user input interface configured to provide user input 
(see figure 4, which recite an input section 35 that provides input for the main control portion 21 
that is responsible for forming the multiplexed media streams). 

For claim 26, Saito discloses a portable communication device configured to form an 
output media stream wherein said device further comprises a multiplexing unit configured to 
provide synchronization of signals of one media type from the first media stream in relation to 
signals of the other media type from the same first media stream, within the output media stream 
(see column 7 lines 23-31, which recite providing synchronization of the video and audio data). 

For claim 27, Saito discloses a portable communication device configured to form an 
output media stream, further comprising a delaying unit configured to provide synchronized 
signals within the output media stream (see column 7 lines 20-31, which recite delaying signals 
of one media type by shifting the value of the Presentation Time Stamp PTS value to a later time 
to provide synchronized signals video and audio signals). 

For claim 28, Saito discloses a portable communication device configured to form an 
output media stream wherein the delaying unit provides synchronization of signals from the 
second media stream, prior to combining with the first stream (see column 7 lines 20-31, which 
recite delaying signals of one media type by shifting the value of the Presentation Time Stamp 
PTS value to a later time to provide synchronized signals video and audio signals). 
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For claim 29, Saito discloses a portable communication device configured to form an 
output media stream wherein the delaying unit provides synchronization of signals of one media 
type in relation to signals of the other media type from the same second media stream (see 
column 7 lines 20-31, which recite delaying signals of one media type by shifting the value of the 
Presentation Time Stamp PTS value to a later time to provide synchronized signals video and 
audio signals). 

10. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Saito (U.S. Patent 
6,987,767) in view of Maclnnis et al. (U.S. Patent 6,853,385) and further in view of Knuutila et 
al. (U.S. Patent 6,810,035). 

For claim 5, Saito and Maclnnis et al. disclose all the subject matter of the claimed 
invention with the exception wherein the established connection is a circuit-switched connection. 
Knuutila et al. from the same or similar fields of endeavor disclose a wireless terminal that 
communicates real time media using a circuit switched connection (see column 2 lines 22-32). 
Thus, it would have been obvious to the person of ordinary skill in the art at the time of the 
invention to use the wireless terminal that establishes circuit-switched connections as taught by 
Knuutila et al. with the portable communication device configured to form an output media 
stream as taught by Saito. The wireless terminal that establishes circuit-switched connections 
can be implemented by using the radio transceiver 204 as taught by Knuutila et al. as the radio 
transceiver of the portable communication device as taught by Saito. The motivation as 
suggested by Knuutila et al. for using the wireless terminal that establishes circuit-switched 
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connections with the portable communication device configured to form an output media stream 
is to provide communication of real time service (see column 2 lines 42-44). 

Response to Arguments 

1 1 . Applicant's arguments with respect to claims 1 , 2, 4-7, and 9-29 have been considered but 
are moot in view of the new ground(s) of rejection. 

Conclusion 

12. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure (please see form PTO-892). 

1 3 . Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to BENH. LIU whose telephone number is (571)270-3118. The 
examiner can normally be reached on 9:00AM to 6:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ricky Ngo can be reached on (571)272-3 139. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Ricky Ngo/ 

Supervisory Patent Examiner, Art Unit 
2464 
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